Skip to content

Default implicit model routing to cost/auto instead of cost/balanced - #76602

Merged
SivaKesava1 merged 2 commits into
mainfrom
copilot/default-implicit-routing-update
Nov 10, 2026
Merged

SivaKesava1 merged 2 commits into
mainfrom
copilot/default-implicit-routing-update

Conversation

Copilot AI commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Eligible workflows that set neither a model nor routing were routed with goal: cost / mode: balanced. This switches the implicit default to mode: auto, so the router picks the mode for each run's task. When classification is unavailable or recommends unknown, auto falls back to balanced.

Only the implicit default changes. Eligibility, model-family restrictions, the runtime model-variable bypass, the engine.model-routing: false opt-out, explicit routing (mode is still required) and main-agent-only scope are unchanged. This is the routing mode auto, not model: auto.

Changes

  • Compiler: defaultModelRoutingMode is now "auto" in pkg/workflow/model_routing_default.go. The single-workflow and batch compile notices read the same constant, so both now say mode auto and remain informational. Comments and schema descriptions that named cost/balanced are updated.

  • Docs: the default claim is updated in:

    • reference/model-routing.md: the default-routing section, two other mentions, and a new sentence on the auto → balanced fallback;
    • reference/engines.md and reference/engines/{copilot,claude,codex,pi}.md;
    • reference/glossary.md and reference/frontmatter-full.md.

    Examples of explicit modes are untouched.

  • Tests:

    • Only the assertions about the implicit default in model_routing_test.go change, plus a new batch-notice assertion. Explicit-mode tests are untouched.
    • WASM golden files are regenerated; the only change in them is balanced → auto.
  • Locks (make recompile): 92 lock files change, matching the 92 locks on main that contain id: resolve-model-routing. Each changed line is only balanced → auto, in three places:

    • the activation message: echo "Model routing enabled: goal cost, mode auto"
    • the firewall config objective: apiProxy.routing.objective = {"goal":"cost","mode":"auto"}
    • the routing run-info metadata in GH_AW_INFO_MODEL_CONFIG

    No other lock content changes (no hash, version or pin drift). No explicitly routed or opted-out lock changes. For the provenance-managed workflows (agentic-token-audit, agentic-token-optimizer), the source Markdown is untouched and only the locks are regenerated.

Changed locks (92)

agent-performance-analyzer, agentic-token-audit, agentic-token-optimizer, architecture-guardian, auditor, aw-issue-clustering, breaking-change-checker, ci-coach, cli-consistency-checker, copilot-opt, copilot-pr-merged-report, copilot-pr-nlp-analysis, copilot-pr-prompt-analysis, craft, daily-agent-of-the-day-blog-writer, daily-ambient-context-optimizer, daily-assign-issue-to-user, daily-compiler-quality, daily-compiler-threat-spec-optimizer, daily-declarative-engine-improver, daily-ecosystem-explorer, daily-experiment-report, daily-formal-spec-verifier, daily-geo-optimizer, daily-graft-intelligence, daily-pr-review-cursor, daily-regression-audit-kiro, daily-report-dispatcher, daily-safe-output-integrator, daily-schema-audit-cursor, daily-secrets-analysis, daily-security-observability, daily-spdd-spec-planner, daily-spec-coverage-kiro, daily-squid-image-scan, daily-syntax-error-quality, daily-team-status, daily-testify-uber-super-expert, daily-trajectory-grader-implementer, daily-work-queue-formal-verification, daily-work-queues-report, daily-workflow-security-model, daily-workflow-updater, dataflow-pr-discussion-dataset, dead-code-remover, delight, deployment-incident-monitor, dev-hawk, dictation-prompt, docs-noob-tester, draft-pr-cleanup, engine-conformance-copilot, engine-conformance-dispatcher, enterprise-docs-audit, eslint-factory-dispatcher, eslint-miner, firewall-escape, firewall, layout-spec-maintainer, mcp-inspector, mergefest, objective-impact-report, org-health-report, plan, pr-finisher, pr-nitpick-reviewer, refiner, release, repository-quality-improver, security-compliance, slide-deck-maintainer, smoke-checkout-pr-dispatch, smoke-copilot-dynamic-workflow, smoke-create-cross-repo-pr, smoke-issues, smoke-multi-pr, smoke-project, smoke-security-alerts, smoke-update-cross-repo-pr, smoke-work-queue-dispatcher, smoke-work-queue-worker, smoke-workflow-call-with-inputs, spec-extractor, spec-librarian, stale-pr-cleanup, tidy, ubuntu-image-analyzer, visual-regression-checker, weekly-blog-post-writer, weekly-editors-health-check, workflow-health-manager, workflow-skill-extractor

Notes for reviewers

  • Known failures on main: in pkg/linters, TestDocSurfacesMatchRegistryAndSpecList and TestDocGo_AnalyzersMatchREADME also fail on the base commit.
  • Timing-sensitive tests: in pkg/cli, TestDownloadWorkflowLogsForTargets{ConcurrentAndResilient,UsesOneWallClockTimeout} hit their time limits during the full test run. They pass when run alone with this change.
  • Not done here: no workflows were dispatched; live branch-run evidence (acceptance case C) is for maintainers. The CodeQL check timed out, so there is no security scan result.

Co-authored-by: SivaKesava1 <11771739+SivaKesava1@users.noreply.github.com>
Copilot AI changed the title [WIP] Update default implicit model routing to cost/auto Default implicit model routing to cost/auto instead of cost/balanced Nov 10, 2026
Copilot AI requested a review from SivaKesava1 November 10, 2026 20:45
@SivaKesava1
SivaKesava1 marked this pull request as ready for review November 10, 2026 21:18
Copilot AI dynamic review requested due to automatic review settings November 10, 2026 21:18
@github-actions

github-actions Bot commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

✅ PR Specification Reviewer completed successfully!

PR Specification Reviewer

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from PR Specification Reviewer · gpt56 · ~2 AIC · ⌖ ~2 AIC · ◇ ~1 AIC · ⊞ 1.6K · ◷

Generated by PR Specification Reviewer for #76602

@github-actions

github-actions Bot commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

✅ PR Code Quality Reviewer completed the code quality review.

🔎 Code quality review by PR Code Quality Reviewer

@github-actions

github-actions Bot commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

✅ Design Decision Gate 🏗️ completed the design decision gate check. See the comment below for the result and any generated ADR draft.

Design Decision Gate 🏗️

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Design Decision Gate 🏗️ · routed: haiku55 xhigh · ~1 AIC · ⌖ ~2 AIC · ◇ ~1 AIC · ⊞ 8.5K · ◷

🏗️ ADR gate enforced by Design Decision Gate 🏗️

@github-actions

github-actions Bot commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

@github-actions

github-actions Bot commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

✅ Test Quality Sentinel completed test quality analysis.

Test Quality Sentinel

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Test Quality Sentinel · routed: haiku55 xhigh · ~1 AIC · ⌖ ~2 AIC · ◇ ~1 AIC · ⊞ 7.2K · ◷

🧪 Test quality analysis by Test Quality Sentinel

@github-actions

github-actions Bot commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

✅ Ponytail Reviewer completed successfully!

Ponytail Reviewer

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Ponytail Reviewer · routed: gpt61 medium · ~5 AIC · ⌖ ~1 AIC · ◇ ~1 AIC · ⊞ 181 · ◷

Generated by Ponytail Reviewer for #76602

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the changed lines and generated lock/golden updates. I found no actionable correctness, performance, or maintainability issues that warrant an inline comment. The change consistently switches only the implicit default from cost/balanced to cost/auto, while explicit routing and opt-out behavior remain unchanged. Review was static; tests were not run per review instructions.

🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt50 · ~3 AIC · ⌖ ~1 AIC · ⊞ 1.6K
Comment /review to run again

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The fleet-wide default change lacks the authorized branch-run evidence required by Issue #76601 acceptance case C to validate auto-mode selection, observed AIC, and task outcomes on the reviewed revision.

No open issues 🎉

What changed in this PR

Updates gh-aw’s implicit model-routing default from cost/balanced to cost/auto, addressing #76601 so eligible workflows use task-specific routing modes. Explicit choices, eligibility rules, runtime bypasses, and opt-outs remain unchanged.

Changes:

  • Switches the compiler default to auto and updates implicit-routing tests, including the batch notice assertion.
  • Aligns documentation and schema descriptions, explaining the existing fallback to balanced.
  • Refreshes generated workflow locks and WASM golden expectations to reflect the new mode.
File Description
pkg/​workflow/​testdata/​TestWasmGolden_CompileFixtures/​with-imports.golden Updates routing notice and metadata expectations.
pkg/​workflow/​testdata/​TestWasmGolden_CompileFixtures/​smoke-copilot.golden Updates routing notice and metadata expectations.
pkg/​workflow/​testdata/​TestWasmGolden_CompileFixtures/​playwright-cli-mode.golden Updates routing notice and metadata expectations.
pkg/​workflow/​testdata/​TestWasmGolden_CompileFixtures/​custom-safe-output-max-length.golden Updates routing notice and metadata expectations.
pkg/​workflow/​testdata/​TestWasmGolden_CompileFixtures/​basic-copilot.golden Updates routing notice and metadata expectations.
pkg/​workflow/​testdata/​TestWasmGolden_AllEngines/​pi.golden Updates Pi routing metadata expectation.
pkg/​workflow/​testdata/​TestWasmGolden_AllEngines/​copilot.golden Updates Copilot routing notice and metadata expectations.
pkg/​workflow/​model_routing.go Updates the implicit-routing field comment.
pkg/​workflow/​model_routing_test.go Checks the new default and batch notice.
pkg/​workflow/​model_routing_default.go Changes the default mode constant to auto.
pkg/​workflow/​engine.go Updates the opt-out field comment.
pkg/​workflow/​compiler_mutators.go Updates the batch-count method comment.
pkg/​parser/​schemas/​main_workflow_schema.json Updates default-routing descriptions without changing validation.
docs/​src/​content/​docs/​reference/​model-routing.md Documents cost/auto and the balanced fallback.
docs/​src/​content/​docs/​reference/​glossary.md Updates the routing default definition.
docs/​src/​content/​docs/​reference/​frontmatter-full.md Updates default and opt-out comments.
docs/​src/​content/​docs/​reference/​engines/​pi.md Documents Pi’s new implicit routing mode.
docs/​src/​content/​docs/​reference/​engines/​copilot.md Documents Copilot’s new implicit routing mode.
docs/​src/​content/​docs/​reference/​engines/​codex.md Documents Codex’s new implicit routing mode.
docs/​src/​content/​docs/​reference/​engines/​claude.md Documents Claude’s new implicit routing mode.
docs/​src/​content/​docs/​reference/​engines.md Updates the shared default-routing description.
.github/​workflows/​smoke-work-queue-worker.lock.yml Sets generated routing notice, metadata, and objective to auto.
.github/​workflows/​smoke-work-queue-dispatcher.lock.yml Sets generated routing notice, metadata, and objective to auto.
.github/​workflows/​smoke-security-alerts.lock.yml Sets generated routing notice, metadata, and objective to auto.
.github/​workflows/​smoke-issues.lock.yml Sets generated routing notice, metadata, and objective to auto.
.github/​workflows/​slide-deck-maintainer.lock.yml Sets generated routing notice, metadata, and objective to auto.
.github/​workflows/​refiner.lock.yml Sets generated routing notice, metadata, and objective to auto.
.github/​workflows/​plan.lock.yml Sets generated routing notice, metadata, and objective to auto.
.github/​workflows/​objective-impact-report.lock.yml Sets generated routing notice, metadata, and objective to auto.
.github/​workflows/​enterprise-docs-audit.lock.yml Sets generated routing notice, metadata, and objective to auto.
.github/​workflows/​daily-team-status.lock.yml Sets generated routing notice, metadata, and objective to auto.

Review effort: Balanced (auto)

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skills-Based Review 🧠

Applied /tdd and /codebase-design. The change is consistent end to end: the defaultModelRoutingMode constant drives both notices, the schema descriptions, the golden files, and the 92 lock files. auto is already a valid mode in model_routing.go. One minor test gap is noted inline.

📋 Key Themes & Highlights

Key Themes

  • Test gap: the batch-notice assertion checks the helper string, not the emitted stderr at pkg/cli/compile_pipeline.go:916.
  • Scope note: the pre-fetched patch omitted the docs and lock-file changes. I reviewed the committed HEAD d0e0548 directly. The docs are updated and all 92 locks show mode auto.

Positive Highlights

  • ✅ Single source of truth for the default, so the code, notice, and tests cannot drift.
  • ✅ Explicit modes, opt-outs, and model-variable bypasses are untouched.
  • ✅ Targeted go test ./pkg/workflow run for the routing default and WASM golden tests passes at HEAD.

Unverified

  • The docs claim that auto falls back to balanced. That fallback lives in the external AWF router and is not implemented in this repo, so I could not verify it here.

@copilot please address the review comment above.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · routed: haiku55 xhigh · ~6 AIC · ⌖ ~2 AIC · ⊞ 6.6K
Comment /matt to run again

@@ -969,21 +969,22 @@ Report a finding.
require.NoError(t, compiled.err)
require.Zero(t, compiled.warnings, "the implicit default must not add warnings")
require.Equal(t, 1, strings.Count(compiled.stderr, defaultRoutingNoticeMarker), "one informational notice per workflow")
require.Contains(t, compiled.stderr, "goal cost and mode balanced")
require.Contains(t, compiled.stderr, "goal cost and mode auto")
require.Contains(t, DefaultModelRoutingBatchNotice(2), "goal cost and mode auto", "batch compilation reports the same default")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/tdd] This assertion only checks the string returned by DefaultModelRoutingBatchNotice, so it would not catch a regression where the aggregated notice stops reaching stderr. The emitter at pkg/cli/compile_pipeline.go:916 has no test in pkg/cli.

💡 Suggested test

Add a batch-compile test in pkg/cli that runs two eligible workflows through the batch pipeline and asserts that stderr contains goal cost and mode auto exactly once. This pins the emitted text, not just the helper.

@copilot please address this.

@SivaKesava1
SivaKesava1 merged commit 041d7c1 into main Nov 10, 2026
154 of 155 checks passed
@SivaKesava1
SivaKesava1 deleted the copilot/default-implicit-routing-update branch November 10, 2026 21:36
@github-actions

Copy link
Copy Markdown
Contributor

🎉 This pull request is included in a new release.

Release: v1.2.3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Default implicit model routing to cost/auto instead of cost/balanced

3 participants